Coding Exercise

In this exercise, you will implement Deep Q-Learning to solve OpenAI Gym's LunarLander environment. To begin, open the Workspace in the next concept, navigate to the exercise/ folder, and follow the instructions in Deep_Q_Network.ipynb.

(Alternatively, if you'd prefer to explore a complete implementation, enter the solution/ folder, and run the code in Deep_Q_Network_Solution.ipynb.)

After you are able to get the code working, try to change the hyperparameters and architecture, to see if you can get the agent to train faster!

## PyTorch

The implementation is written in PyTorch. If the PyTorch framework is new to you, please take a look at our Deep Learning with PyTorch lesson in the extracurricular content.

## Note

In the Workspace in the following concept, you will have the option to ENABLE GPU to accelerate training. After training, you can use the provided code in the Jupyter notebook to watch your agent's performance. Note that if visualizing the trained agent in the Workspace, GPU should be disabled -- otherwise the notebook will return an error.

Thus, you are encouraged to follow the following workflow:

train the agent with GPU enabled, and save the trained model weights,
disable GPU, load the trained weights from file, and watch the trained agent.

## DQN Improvements

Later in this lesson, you will learn about three different improvements you can make to your algorithm:

Double DQN
Prioritized Experience Replay
Dueling DQN

If you'd like to implement any of these, you're encouraged to use the provided notebook as a starting point.